imageSVM - A USER-ORIENTED TOOL FOR ADVANCED CLASSIFICATION OF HYPERSPECTRAL DATA USING SUPPORT VECTOR MACHINES

An implementation for the classification of remote sensing images with support vector machines (SVM) is introduced. This tool, called imageSVM, allows a user-friendly work, especially with large, highly-resolved data sets in the ENVI/IDL environment. imageSVM uses LIBSVM for the training of the SVM in combination with a user-defined grid search. Parameter settings can be set flexibly during the entire workflow and a time-efficient processing becomes possible. First tests underline the high-accuracy of SVM classification using heterogeneous hyperspectral data and the good performance of SVM in the context of multi-sensoral studies. INTRODUCTION Over the past two decades, a variety of non-parametric classifiers has been introduced into remote sensing image analysis, e.g. artificial neural networks (i), decision tree classifiers (ii), and support vector machines (iii,iv). SVM do not assume specific class distributions and are well suited for complex hyperspectral data sets (v). They are one of the more recent developments in the field of machine learning and outperformed other approaches under varying conditions in the very most cases or performed at least equally well (vi,vii), especially when small numbers of training samples were used. However, the processing of image data with latest machine learning developments for the SVM classification is often complicated due to missing user interfaces and inappropriate data formats like ASCII. In addition, the processing of large, highly resolved data sets often leads to long processing times. Furthermore, the classification of remote sensing images usually includes more than two classes and strategies for the originally binary SVM have to be applied to solve multi-class problems (iv,viii). First implementations of SVM in commercial image processing software packages are rather limited in performance, especially in the context of searching ideal SVM parameters or optimized training and classification procedures. This work aimed at developing a flexible SVM implementation for remote sensing image analysis called imageSVM. Objectives for the design were (1) offering the full flexibility of state-of-the-art algorithms for the training of the SVM, (2) the possibility to use standard remote sensing formats, (3) a semi-automized and time-saving search for optimal training parameters, (4) the possibility for a quick validation of the classification accuracy for methodological studies, and (5) SVM models and the intermediate results of the classification have to be flexibly accessible, e.g. for data fusion or classifier ensemble approaches. METHODS AND IMPLEMENTATION SVM delineate two classes by fitting an optimal separating hyperplane to the training data in the multi-dimensional feature space (ix). For linearly not separable classes, the input data are mapped into a higher dimensional space by a kernel function, wherein the new data distribution enables the fitting of a linear hyperplane. A detailed description on the concept of SVM is given in (x), comprehensive introductions in a remote sensing context in (iii,iv). Two parameters need to be set for the training of SVM with Gaussian kernels: the parameter γ that controls the width of the kernel and a Proceedings 5 EARSeL Workshop on Imaging Spectroscopy. Bruges, Belgium, April 23-25 2007 2 regularization parameter, either C or ν, depending on the preferred SVM formulation. The regularization parameter C controls the trade-off between maximizing the margin and the penalization of training errors (more precisely margin errors). ν is an upper bound on the fraction of training errors and a lower bound of the fraction of support vectors (xi). Besides the Gaussian kernel, any kernel function that meets the Mercer condition (ix) can be used with SVM, e.g. polynomials or sigmoidal functions. Since Gaussian kernels are most frequently used, only these were implemented in imageSVM. The widely accepted LIBSVM approach by (xii) was integrated in the ENVI/IDL Environment for the training of the SVM. This way the training and classification can be performed on common image file formats. Samples for training and validation are taken from ROIs. Both SVM-types, Cor ν-SVM, and the two common multi-class strategies one-against-all and one-against-one can be selected (Fig. 1, left). Figure 1: dialogue for SVM formulation and multi-class strategy (left) and for grid search and cross validation settings (right). To determine an optimal set of the parameters γ and C or ν a grid search with internal cross validation is performed. Besides the ranges and step-size of the parameter search the user can define the number of folds for the cross validation (CV) and the tolerance for the termination criterion of the quadratic optimization during the cross-validation trainings (Fig. 1, right). All parameters have great influence on the time needed for the calculation of the grid search and thus the determination of appropriate parameters is very important. The CV results for all parameter combinations during the grid search are saved as images. These so-called CV images can be used to assess the quality of the grid search and to identify the ideal parameters for the training of the final SVM (Fig. 2). The user can choose, whether the best parameters shall be selected individually for each binary SVM or if one best overall parameter set from a mean CV error image shall be used for the final training of all binary classifiers. This final training is independent from the previous grid search and the number of features, samples or the termination criterion can be changed compared to the grid search. This way, a time saving calculation during grid search is performed while equal final accuracy is achieved. The results from the final training with the best parameter settings are saved as so-called SVM models. These can be viewed in the imageSVM model viewer, which shows the training parameters and gives information on the data used during the training (Fig. 3). From the model viewer the user can choose between the classification of entire images or a quick validation, where only reference pixels from an ROI are classified and used for an accuracy assessment. Proceedings 5 EARSeL Workshop on Imaging Spectroscopy. Bruges, Belgium, April 23-25 2007 3